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(54) Blind source separation for hearing aids 

(57) An electronic filtering device for performing re- 
al-time unmixing of a signal desired to be recovered by 
a user of the device, where the desired signal emanates 
from one of a plurality of independent signal sources. 
Two microphones positioned along a common axis de- 
velop first and second electrical input signals in re- 
sponse to reception by the microphones of acoustic sig- 
nals from the plurality of independent signal sources. 
The common axis of the microphones is controllable in 
real time by the user to align the common axis so it points 
in the direction of the source of the desired signal. An 



adaptive unmixing signal processor responsive to the 
input signals develops output signals wherein the de- 
sired signal is separate from the mixture signal. A pre- 
processor may be provided to subject the input signals 
to one or both of a time delay processing and a decor- 
relation processing before their application to the unmix- 
ing signal processor, to enhance recovery of the desired 
signal. A selected output of the unmixing signal proces- 
sor can be applied as an input to a speaker for repro- 
duction, or can be further processed for signal enhance- 
ment by an additional processor before reproduction. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention s 

[0001] The present invention generally relates to elec- 
tronic filtering for enhancing a desired signal component 
of a mixed signal, and more specifically to a method and 
apparatus for real-time unmixing (separation or decon- 
volving) of a desired signal from a mixture of independ- 
ent signals, particularly useful, for example, in a hearing 
aid. 

2. Description of the Prior Art 

[0002] When one is listening to someone or some- 
thing, "noise" or undesired signals that interfere with the 
voice or desired signal, are ubiquitous. People with 
hearing impairment are especially vulnerable to noise. 
Background conversations, interference from digital de- 
vices (mobile telephones), car, or other specific environ- 
ment noises, can make it very difficult for a hearing im- 
paired person to understand a desired speech signal. A 
reduction in the noise level of a signal, coupled with an 
automatic focus on a desired signal component, can sig- 
nificantly improve the performance of an electronic 
voice processor, such as one used in an advanced hear- 
ing aid. 

[0003] In recent years, hearing aids using digital sig- 
nal processing have been introduced. They contain one 
or more microphones, analog to digital converters, dig- 
ital signal processors, and speakers. Usually the digital 
signal processors divide the incoming signals into sev- 
eral frequency regions using filter banks. Within each of 
those regions, signal gain and dynamic compression 
parameters can be individually adjusted in accordance 
with the requirement for a particular user of the hearing 
aid, in an attempt to improve intelligibility. Additionally, 
digital signal processing algorithms for feedback reduc- 
tion and noise reduction are available, however they 
have major limitations. For example, some of the disad- 
vantages of the currently available algorithms for noise 
reduction are the limited improvement they obtain when 
speech and background noise are in the same frequen- 
cy region, due to their inability to distinguish between 
speech and background noise. 

[0004] One relatively new digital signal processing 
approach currently finding use for noise reduction in ar- 
eas such as speech recognition, data communication 
and sensor signal processing, involves a technique 
known generally as Independent Component Analysis 
(ICA), and in more specific applications as Blind Source 
Separation (BSS). This technique searches an input sig- 
nal having multiple components, for a signal transfor- 
mation which will minimize the statistical dependence 
between its components. Accordingly, BSS is a signal 
separation technique capable of delivering dramatic im- 



provements in signal to noise ratio for mixtures of inde- 
pendent signals, such as multiple voices or mixtures of 
voice and noise signals. 

[0005] It is an object of the present invention to pro- 
vide an electronic filtering technique incorporating BSS 
processing which can operate in real time to enhance 
reception of a desired signal, such as the voice of a near- 
by person, and furthermore, if desired, can be incorpo- 
rated in a hearing aid. 

SUMMARY OF THE INVENTION 

[0006] An electronic filtering device for performing re- 
al-time unmixing of a signal desired to be recovered by 
a user of the device, where the desired signal emanates 
from one of a plurality of independent signal sources. 
Two microphones positioned along a common axis de- 
velop first and second electrical input signals in re- 
sponse to reception by the microphones of acoustic sig- 
nals from the plurality of independent signal sources. 
The spatial position of the common axis of the micro- 
phones is controllable in real time by the user to align 
the common axis so it points in the direction of the 
source of the desired signal, thereby imparting an inher- 
ent directionality to the input signals. An adaptive un- 
mixing signal processor responsive to the input signals 
develops output signals wherein the desired signal is 
separated from the mixture signal. In one preferred em- 
bodiment of the invention a preprocessor is provided to 
enhance the inherent directionality of the input signals 
by establishing a relative time delay therebetween. Fur- 
thermore, the preprocessor may subject the enhanced 
input signals to a decorrelation processing before their 
application to the unmixing signal processor. A selected 
output of the unmixing signal processor can be applied 
as an input to a speaker for reproduction, or can be fur- 
ther processed for signal enhancement by an additional 
processor before reproduction. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0007] 

Figure 1 illustrates in block diagram form an elec- 
tronic filtering device constructed in accordance 
with the principles of the present invention; 

Figure 2 illustrates in block diagram form the pre- 
processing stage of the electronic filtering device 
shown in Figure 1 ; 

Figure 3 illustrates in block diagram form the tech- 
nique of Blind Source Separation as used in the 
electronic filtering device of the invention; and 

Figure 4 illustrates in block diagram form an exem- 
plary embodiment of a Blind Source Separator use- 
ful in the electronic filtering device of the invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

[0008] Figure 1 illustrates in block diagram form an 
application of the invention for use in hearing aids. A 
hearing aid 1 0 includes two microphones 1 2 and 1 4 for s 
developing two input signals 1 and 2, respectively. In 
accordance with one aspect of the invention, the micro- 
phones are mounted in the hearing aid such that a com- 
mon axis of their positioning always extends substan- 
tially in the direction in which the wearer of the hearing io 
aid looks when being attentive to a signal source such 
as a voice. This microphone positioning imparts an in- 
herent directionality to input signals 1 and 2. Since each 
microphone develops electrical signals representative 
of the acoustic waves received thereby from sound is 
sources within it's operating range, each input signal 
may comprise a mixture of unknown signals from an un- 
known number of signal sources. Input signals 1 and 2 
are processed in three main stages. At a first stage 16, 
the input signals are preprocessed for enhancing the in- 20 
herent directionality already imparted thereto by their 
positioning. At a second stage 18, the resulting signals 
are subjected to an unmixing processing (sometimes re- 
ferred to as separation processing), which is designed 
to produce estimates of the original unknown signals 25 
picked-up by microphones 12 and 14. At a third stage 
20, the outputs of the unmixing processing are prefera- 
bly postprocessed to produce the desired signal 22, 
which can then be applied to a speaker 24 of the hearing 
aid 10 for reproduction and presentation to a user. 30 
[0009] As illustrated in Fig. 2, preprocessing stage 1 6 
begins with normalization of the raw input signals. Au- 
tomatic Gain Control is used to normalize input signals 
1 and 2 to a [-1 ,+1] range. The inputs 1 and 2 are now 
given in by a vector x = (x 1 (t),x 2 (t)). 35 
[001 0] In accordance with one aspect of the invention, 
in order to adapt a blind source separation (BSS) tech- 
nique for use in a device as small as a hearing aid, and 
to have it operate in real-time, preprocessing stage 16 
also provides at least the first, and preferably both of the *o 
following additional processing: 

• Enhancement of signal source directionality inher- 
ent in the input signals, resulting from a directional 
arrangement of microphones 12 and 14 with re- 45 
spect to a source of interest. In the hearing aid ex- 
emplary embodiment, the directionality of the 
source of interest is presumed to be in the direction 
that the user is looking. Accordingly, the micro- 
phones are positioned on the hearing aid along an so 
axis that is in the direction that the user would be 
looking, and the direction of the source of interest 
is presumend to be at zero degrees with respect to 
such axis. The direction of a second source can be 
estimated in the preprocessing stage (delay box in 55 
16) resulting in an adaptive delay (5). The delay is 
a positive or negative fractional delay, such that the 
most powerful component of the inputs other than 



the one approximately aligned with the microphone 
axis arrives synchronously at the two microphones. 
For example th is would be zero if the second source 
were perpendicular to the microphone axis. For this 
enhancement, the normalized input signals x = (x 1 
(t),x 2 (t)) are modified as follows: 

Xi(t)=x,(t) 



x 2 (t)=x 2 (t-b) 

* Decorrelation of the input signals. In the exemplary 
embodiment decorrelation is carried out by a diag- 
onalization of the correlation matrix. More specifi- 
cally, let C=Covariance{ x T ), where x T is a trans- 
pose of x. If significant correlation exists between 
the two input signals (x-,, x 2 ), a decorrelation over a 
time window D means transformation of the signals 
in two steps: (1) centering around the mean over 
the data in the window D; and (2) Affine transforma- 
tion of the resulting data points in order to diagonal- 
ize the covariance matrix of the resulting signals. 
Assuming that x is centered around its mean, we 
use the following transformation: 

x=2(*/C)~ 1 x 

[0011] In the illustrated embodiment, the window D 
comprised 16,000 samples. 

[001 2] The above described preprocessing facilitates 
the subsequent BSS processing to arrive at a solution 
in a shorter time than if the preprocessing was not pro- 
vided, and furthermore, increases the probability that 
the BSS processing will arrive at a valid solution instead 
of a local minimum. 

[001 3] Figure 3 illustrates the principles of the opera- 
tion of a BSS algorithm upon which the unmixing or sep- 
aration of the desired component from the input signals 
is based. The technique is called Blind Source Separa- 
tion because it makes few assumptions about the type 
of signals present in the mixture. As well known by those 
of ordinary skill in this technology, BSS processing is 
intended to recover the set of n unknown source signals 
from a set of their mixtures, assuming that the n source 
signals are independent. More specifically, as shown in 
Figure 3, if s is a vector of n sources, and x is a vector 
of m observations of those sources (i.e., the raw input 
signals from the m microphones), the goal of a BSS 
processor is to discover the m by n mixing matrix A: 

x= As .where x is the preprocessed signals shown 
in Figure 2 (i.e., x"). 

or equivalently, and as is done in the present invention, 
to find an unmixing or separating matrix W such that 

z - Wx = s = s where z is the vector of the inde- 
pendent estimates of component signals s and z is an 
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estimate of the source signals. 

[0014] As previously noted, the sources s=(s p s^land 
the environment-dependent mixing matrix A are un- 
known. The BSS processor (which as well known, may 
be implemented using a neural network) only sees the 
inputs x=(x p X2) coming from two microphones in order 
to determine estimates z=(z 1t of the independent 
component signals s. In this case, the inputs x are ac- 
tually the preprocessed signals x\ previously de- 
scribed. 

[001 5] Figure 4 illustrates a block diagram of the main 
components of a BSS processor 400. BSS processor 
400 comprises: an unmixing component 402 for record- 
ing and updating the state of the unmixing process de- 
fined by parameters W and v; a nonlinear component 
404 for generating statistics used in the adaptation proc- 
ess; and an adaptation component 406 for computing 
changes in the values of the unmixing parameters, AW 
and Av. 

[0016] As will now be described in greater detail, the 
BSS processor 400 continuously adapts two state vari- 
ables: the 2 by 2 unmixing matrix W, and the 2 by 1 bias 
vector b. The unmixing component 402 buffers the most 
recent N samples input to BSS processor 400. It com- 
putes the output z corresponding to the most recent in- 
put sample x by using the current values of the param- 
eters W. These parameters are initialized with small ran- 
dom values at the beginning of the process (while v=0): 

z=Wx 

[0017] The nonlinear component 404 transforms the 
output of the system using an invert ible mapping. The 
objective of component 404 is to avoid processing very 
large numeric values of the outputs, which may be in- 
finities from a computational point of view. This objective 
is carried out by processing statistically equivalent 
quantities, obtained after running the outputs z through 
the invertible mapping. An example of a nonlinear trans- 
formation used in component 404 is the sigmoidal non- 
linearity y, defined below, taking as arguments z trans- 
lated with v over the input buffer. 

_ 1 
y ~ 1 + exp(-z-v) 

[0018] The adaptation component 406 determines 
changes in the unmixing parameters W and v: i.e., AW 
and Av. The objective is to maximize the mutual infor- 
mation that the outputs y contain about the inputs x, as 
well known to those skilled in this technology, and as 
described, for example by A.J. Bell and T.J. Sejnowski 
in their article entitled "An information-maximization ap- 
proach to blind separation and blind deconvolution" 
published in Neural Computation, 7:1129-1159, 1995, 
and as also described in Bell's US patent 5,706,402. 
Th is objective reduces to a condition on the joint entropy 



H=H(y p y 2 ) of the outputs y; 

dH(y v y 2 ) 



' £ =0 

dv 

10 

[0019] The resulting adaptations rules are modified to 
perform a "natural gradient" step known by those skilled 
in this technology, such as described by S. Amari in his 
publication entitled "Minimum mutual information blind 
is separation, published in Neural Computation, 1996. 
[0020] We obtain the following update rules: 

AW = T|(W+(1-2y).u) 

20 

Av= ti(1 - 2y) 

[0021] A typical value for the learning rate t| is 0.005. 

25 [0022] Referring again to Figure 1 , following unmixer 
18 is the postprocessing step 20, wherein a determina- 
tion is made of which output estimate of unmixer 18 is 
more likely to represent voice rather than noise, as well 
as a normalization of the power of the outputs by scaling 

30 them to the level of the input powers. The output signal 
section can be based on multiple criteria using, for ex- 
ample, voice specific feature extraction and analysis, 
and/or dominant speaker detection, which can also be 
accomplished using feature extraction and analysis. 

35 [0023] As previously noted, in the illustrated embodi- 
ment of the present invention, the BSS processing is ap- 
plied for use in hearing aids. The inputs to the system 
are given by two microphones which, with the present 
invention, can be situated very close to one another. In 

40 terms of the notation in the BSS processor shown in Fig- 
ures 3 and 4, the system has two inputs and two ouputs 
(n=m=2). 

[0024] Particularly for the case of hearing aids, the 
present invention addresses the following problems: 

45 

o It works with real world mixtures of signals in ane- 
choic environments. The challenge is that a hearing 
aid using BSS would incorporate two microphones 
which, given the physical limitation imposed by in 
so the ear hearing aids, may be less than 1 1 mm apart. 
° It can cope with more signals than the number of 
microphones. Until now, this was thought to be im- 
possible since the existing theory behind BSS guar- 
antees that a solution exists only when n>m. 
55 p it works under non-stationary mixing conditions in 
order to follow moving sources and adapt to chang- 
ing listening environments. 
• It works in real time so that the user is not subjected 
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to disconcerting delays in the signals and so that 
the hearing aid can adapt as necessary. 

[0025] Thus, there has been shown and described a 
novel method and apparatus for real-time unmixing of a 
desired signal from a mixture of independent signals. 
Many changes, modifications, variations and other uses 
and applications of the subject invention will, however, 
become apparent to those skilled in the art after consid- 
ering this specification and its accompanying drawings, 
which disclose a preferred embodiment thereof. For ex- 
ample, although pre- and post- BSS processors 16 and 
18 are described, as noted herein, they are not strictly 
necessary in the broadest application of the present in- 
vention. Additionally, the various components of BSS 
processor 400 can be biased with a priori knowledge 
about the input signals to facilitate its operation, for ex- 
ample, knowledge about the distribution of the ampli- 
tude values of the source signals or even that one input 
signal represents speech. Furthermore, signal process- 
ing for enhancing source signal directionality can be in- 
corporated into preprocessor 16. Even furthermore, the 
teaching of the present invention can be extremely use- 
ful for interference cancellation, separation of one voice 
from a mixture of many voices ("cocktail party" problem), 
and for preprocessing sound mixtures for noise reduc- 
tion in order to allow further processing of a desired 
sound signal, x. All such changes, modifications, varia- 
tions and other uses and applications which do not de- 
part from the teachings herein are deemed to be cov- 
ered by this patent, which is limited only by the claims 
which follow as interpreted in light of the foregoing de- 
scription. 



Claims 

1. An electronic filtering device for performing real- 
time unmixing of a signal desired to be recovered 
by a user of the device, where the desired signal 
emanates from one of a plurality of independent sig- 
nal sources, comprising: 

two microphones positioned along a common 
axis for developing first and second electrical 
input signals in response to reception by the mi- 
crophones of acoustic signals from the plurality 
of independent signal sources, wherein the 
spatial position of the common axis of the mi- 
crophones is controllable in real time by the us- 
er to align the common axis so that it substan- 
tially continuously points in the direction of the 
source of the desired signal; and 
an adaptive unmixing signal processor respon- 
sive to said input signals for developing output 
signals wherein the desired signal is separate 
from the mixture signal. 



2. The apparatus of claim 1 , wherein the common axis 
is positioned on the user in a manner so as to point 
in the direction of the source. 

s 3. The apparatus of claim 2, wherein said micro- 
phones are mounted in a common housing that is 
intended to be co-located with the ear of the user. 

4. The apparatus of claim 1, further including a pre- 
io processor for modifying the input signals before 

they are applied to the unmixing signal processor. 

5. The apparatus of claim 4, wherein the preprocessor 
introduces a relative delay between components of 

is the input signals. 

6. The apparatus of claim 4, wherein the preprocessor 
subjects the input signals to a decorrelation 
processing. 

20 

7. The apparatus of claim 1 , further including a post- 
processor responsive to the output signals of the 
unmixing signal processor for selecting the desired 
signal for application to a signal reproduction de- 

25 vice. 

8. The apparatus of claim 1 , wherein the unmixing sig- 
nal processor comprises a blind source signal sep- 
arator. 

30 

9. The apparatus of claim 8, wherein the blind source 
signal separator comprises a neural network for 
performing an unsupervised learning process that 
operates to maximize the joint output entropy of the 

35 output signals. 

1 0. A method for performing real-time unmixing of a sig- 
nal desired to be recovered by a user, where the 
desired signal emanates from one of a plurality of 

40 independent signal sources, the method compris- 
ing the following steps: 

positioning two microphones along a common 
axis, for developing first and second electrical 

4S input signals in response to reception by the mi- 

crophones of acoustic signals from the plurality 
of independent signal sources, said positioning 
being such that the common axis of the micro- 
phones is controllable in real time by the user 

50 to align the common axis so that it substantially 

continuously points in the direction of the 
source of the desired signal; and 
subjecting said input signals to an adaptive un- 
mixing signal processing for developing output 

55 signals wherein the desired signal is separated 

from the mixture signal. 

11. The method of claim 10, wherein said positioning 
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locates the common axis proximate the user in a 
manner so that it points in the direction that the user 
is looking. 

12. The method of claim 11, wherein said positioning 
locates the common axis on a common housing that 
is intended to be co-located with the ear of the user. 

1 3. The method of claim 1 0, further including a preproc- 
essing step for modifying the input signals before 
they are subjected to the unmixing signal process- 
ing. 

14. The method of claim 10, wherein the preprocessor 
step introduces a relative delay between the input 
signals. 

1 5. The method of claim 1 4, wherein the preprocessing 
step subjects the relatively delayed input signals to 
decorrelation processing. 

16. The method of claim 15, wherein the decorrelation 
processing step is carried out by a diagonalization 
of a correlation matrix formed using the relatively 
delayed input signals. 

17. The method of claim 10, further including a post- 
processing step responsive to the output signals of 
the unmixing signal processing step for selecting 
the desired signal for application to a signal repro- 
duction device. 



ing for developing output signals wherein the 
desired signal is separated from the mixture 
signal. 

5 21 . The method of claim 20, wherein said positioning is 
such that the common axis of the microphones is 
controllable in real time by the user to align the com- 
mon axis so that it substantially continuously points 
in the direction of the source of the desired signal. 

10 

22. The method of claim 20, wherein said preprocess- 
ing comprises introducing a relative delay between 
the input signals so as to further enhance their di- 
rectionality. 

is 

23. The method of claim 22, wherein said preprocess- 
ing also includes a decorrelation processing of the 
relatively delayed input signals. 



18. The method of claim 10, wherein the unmixing sig- 
nal processing step comprises blind source signal 
separation processing. 35 

19. The method of claim 18, wherein the blind source 
signal separation processing comprises an unsu- 
pervised learning process that operates to maxi- 
mize the joint output entropy of the output signals. *o 



20. A method for performing real-time unmixing of a sig- 
nal desired to be recovered by a user, where the 
desired signal emanates from one of a plurality of 
independent signal sources, the method compris- 
ing the following steps: 



positioning two microphones along a common 
axis, for developing first and second electrical 
input signals in response to reception by the mi- so 
crophones of acoustic signals from the plurality 
of independent signal sources; 
preprocessing the first and second electrical in- 
put signals so as to enhance signal source di- 
rectionality inherent therein due to the position- S5 
ing of the microphones; and 
subjecting said directionality enhanced input 
signals to an adaptive unmixing signal process- 
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